{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MNIST digit classification in TensorFlow 2.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we will see how can we perform the MNIST handwritten digits classification using\n",
    "tensorflow 2.0. It hardly a few lines of code compared to the tensorflow 1.x. As we learned,\n",
    "tensorflow 2.0 uses as keras as its high-level API, we just need to add tf.keras to the keras\n",
    "code."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import the libraries:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.0.0\n"
     ]
    }
   ],
   "source": [
    "print(tf.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load the dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "mnist =  tf.keras.datasets.mnist"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a train and test set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz\n",
      "11493376/11490434 [==============================] - 64s 6us/step\n"
     ]
    }
   ],
   "source": [
    "(x_train,y_train), (x_test, y_test) = mnist.load_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Normalize the x values by diving with maximum value of x which is 255 and convert them to float:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train, x_test = tf.cast(x_train/255.0, tf.float32), tf.cast(x_test/255.0, tf.float32)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "convert y values to int:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_train, y_test = tf.cast(y_train,tf.int64),tf.cast(y_test,tf.int64)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define the sequential model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = tf.keras.models.Sequential()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Add the layers - We use a three-layered network. We apply ReLU activation at the first two layers and in the final output layer we apply softmax function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add(tf.keras.layers.Flatten())\n",
    "model.add(tf.keras.layers.Dense(256, activation=\"relu\"))\n",
    "model.add(tf.keras.layers.Dense(128, activation=\"relu\"))\n",
    "model.add(tf.keras.layers.Dense(10, activation=\"softmax\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compile the model with Stochastic Gradient Descent, that is 'sgd' (we will learn about this in the next chapter) as optimizer and sparse_categorical_crossentropy as loss function and with accuracy as a metric:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(optimizer='sgd', loss='sparse_categorical_crossentropy', metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Train the model for 10 epochs with batch_size as 32:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 60000 samples\n",
      "Epoch 1/10\n",
      "60000/60000 [==============================] - 7s 121us/sample - loss: 0.5882 - accuracy: 0.8490\n",
      "Epoch 2/10\n",
      "60000/60000 [==============================] - 5s 80us/sample - loss: 0.2747 - accuracy: 0.9221\n",
      "Epoch 3/10\n",
      "60000/60000 [==============================] - 5s 78us/sample - loss: 0.2232 - accuracy: 0.9371\n",
      "Epoch 4/10\n",
      "60000/60000 [==============================] - 5s 75us/sample - loss: 0.1900 - accuracy: 0.9464\n",
      "Epoch 5/10\n",
      "60000/60000 [==============================] - 4s 72us/sample - loss: 0.1660 - accuracy: 0.9528\n",
      "Epoch 6/10\n",
      "60000/60000 [==============================] - 5s 84us/sample - loss: 0.1471 - accuracy: 0.9579\n",
      "Epoch 7/10\n",
      "60000/60000 [==============================] - 5s 78us/sample - loss: 0.1323 - accuracy: 0.9624\n",
      "Epoch 8/10\n",
      "60000/60000 [==============================] - 4s 75us/sample - loss: 0.1197 - accuracy: 0.9660\n",
      "Epoch 9/10\n",
      "60000/60000 [==============================] - 5s 79us/sample - loss: 0.1089 - accuracy: 0.9689\n",
      "Epoch 10/10\n",
      "60000/60000 [==============================] - 6s 97us/sample - loss: 0.0999 - accuracy: 0.9719\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f4a7c2d5fd0>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(x_train, y_train, batch_size=32, epochs=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Evaluate the model on test sets:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.evaluate(x_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That's it! Writing code with Keras API is that simple.\n",
    "\n",
    "From the next chapters, we will use TensorFlow 2.0"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
